{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Matrix Algebra using Python\n",
    "\n",
    "Authors <a href=\"mailto:a.owen@leeds.ac.uk\"> Dr Anne Owen </a> and <a href=\"mailto:j.busch@leeds.ac.uk\"> Dr Jonathan Busch </a>\n",
    "\n",
    "Before we start anything interesting, paste the two lines of code below into the box\n",
    "to import the numpy library and to set some display settings. Remember to press Shift-Return to run the cell.\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "%config InteractiveShell.ast_node_interactivity='last_expr_or_assign'\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "%config InteractiveShell.ast_node_interactivity='last_expr_or_assign'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# 1. Definitions and getting started\n",
    "\n",
    "A <b>matrix</b> is a collection of numbers ordered by rows and columns. This is a convenient way to store lots of data. One example of such data could be the scores of several students (rows) on several exams (columns). We usually enclose the numbers in a matrix inside square brackets:\n",
    "\n",
    "$$\n",
    "\\textbf{X}=\n",
    "\\begin{bmatrix} \n",
    "5 & 8 & 2 \\\\\n",
    "1 & 0 & 7 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "## Exercise 1.1: Make a matrix (i)\n",
    "\n",
    "Look at how we construct matrix $\\textbf{X}$ in Jupyter:\n",
    "\n",
    "```python\n",
    "X = np.array([[5, 8, 2], [1, 0, 7]])\n",
    "```\n",
    "\n",
    "We are using the array function from the numpy library. Try pasting the command to create $\\textbf{X}$ into the box below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5, 8, 2],\n",
       "       [1, 0, 7]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.array([[5, 8, 2], [1, 0, 7]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\textbf{X}$ is now a variable that we can refer to and use later. Try changing the numbers in the code above and running the line of code to make a new version of $\\textbf{X}$. When you have finished experimenting, change $\\textbf{X}$ back to the original version.\n",
    "\n",
    "## Exercise 1.2: Make a matrix (ii)\n",
    "\n",
    "What do you need to type to create \n",
    "\n",
    "$$\n",
    "\\textbf{Y}=\n",
    "\\begin{bmatrix} \n",
    "3 & 1 \\\\\n",
    "6 & 7 \\\\\n",
    "2 & 3 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Make $\\textbf{Y}$ in the box below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[3, 1],\n",
       "       [6, 7],\n",
       "       [2, 3]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y = np.array([[3,1], [6,7], [2,3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1.3 using 'size'\n",
    "$\\textbf{X}$ is a 2-by-3 matrix. It has two rows and 3 columns.\n",
    "\n",
    "Use the np.size function to find out the dimensions of $\\textbf{X}$ and $\\textbf{Y}$. \n",
    "\n",
    "Try ```np.size(X,0)``` and ```np.size(X,1)```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.size(X,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```np.size(X,0)``` gives you the dimensions in the vertical direction i.e. the matrix has 2 rows.\n",
    "\n",
    "```np.size(X,1)``` gives you the dimensions in the horizontal direction i.e. the matrix has 3 columns.\n",
    "\n",
    "## Exercise 1.4 Locating a matrix element\n",
    "\n",
    "$$\n",
    "\\textbf{X}=\n",
    "\\begin{bmatrix} \n",
    "5 & 8 & 2 \\\\\n",
    "1 & 0 & 7 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The elements of a matrix are numnbered in the following way:\n",
    "\n",
    "$$\n",
    "\\textbf{X}=\n",
    "\\begin{bmatrix} \n",
    "x_{00} & x_{01} & x_{02} \\\\\n",
    "x_{10} & x_{11} & x_{12} \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "In python you can refer to a cell within a matrix using its row and column number. For example, ```X[0,1]``` returns the answer $8$ because it is the number in the first row and second column.\n",
    "\n",
    "What command returns the answer $7$ from the matrix $\\textbf{X}$?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[1,2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1.5 Matrix types and the 'diag' function\n",
    "\n",
    "A <b>vector</b> is a special type of matrix that only has one row or one column. Below, $\\textbf{a}$ is a row vector and $\\textbf{b}$ is a column vector:\n",
    "\n",
    "$$\n",
    "\\textbf{a}=\n",
    "\\begin{bmatrix} \n",
    "2 & 7 & 4 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\textbf{b}=\n",
    "\\begin{bmatrix} \n",
    "7 \\\\\n",
    "2 \\\\\n",
    "3\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "A <b>scalar</b> is a matrix with a single row and a single column (just one number). \n",
    "\n",
    "By convention, scalars are italicised (e.g. $f$), vectors are lower case and bold (e.g. $\\textbf{a}$) and matrixes are upper case and bold (e.g.$\\textbf{X}$).\n",
    "\n",
    "A <b>square matrix</b> has as many rows as it has columns. $\\textbf{S}$ is an example of a 3 rows by 3 columns (or 3-by-3) square matrix:\n",
    "\n",
    "$$\n",
    "\\textbf{S}=\n",
    "\\begin{bmatrix} \n",
    "4 & 5 & 2 \\\\\n",
    "7 & 1 & 0 \\\\\n",
    "2 & 9 & 15 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The diagonal elements of a matrix are those where the column and row number are equal. The diagonal elements of $\\textbf{S}$ are 4, 1, and 15.\n",
    "\n",
    "In a <b>diagonal matrix</b> all the off-diagonal elements are 0.\n",
    "\n",
    "To create the row vector $\\textbf{a}$ below we use ```a = np.array([[2,7,4]])``` note that we use double square brackets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 7, 4]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([[2,7,4]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now make the column vector $\\textbf{b}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[7],\n",
       "       [2],\n",
       "       [3]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b = np.array([[7], [2], [3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function ```np.diagflat()``` can be used to turn vectors into diagonal matrices. Try it out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 0, 0],\n",
       "       [0, 7, 0],\n",
       "       [0, 0, 4]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.diagflat(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1.6 The identity matrix\n",
    "\n",
    "An <b>identity</b> matrix is a diagonal matrix containing only $1$s on the diagonal. Identity matrices are denotes as $\\textbf{I}$\n",
    "\n",
    "To create \n",
    "$\\textbf{I}=\n",
    "\\begin{bmatrix} \n",
    "1 & 0 & 0 \\\\\n",
    "0 & 1 & 0 \\\\\n",
    "0 & 0 & 1 \n",
    "\\end{bmatrix}\n",
    "$ in python we use the ```np.identity``` function\n",
    "\n",
    "Try ```I = np.identity(3)```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "I = np.identity(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1.7 Matrices of zeros\n",
    "\n",
    "To create \n",
    "$\\textbf{D}=\n",
    "\\begin{bmatrix} \n",
    "0 & 0 & 0 \\\\\n",
    "0 & 0 & 0 \n",
    "\\end{bmatrix}\n",
    "$ in python we use the ```np.zeros``` function\n",
    "\n",
    "Try ```D = np.zeros((2,3))```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0.],\n",
       "       [0., 0., 0.]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "D = np.zeros((2,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Matrix addition and subtraction\n",
    "\n",
    "To add and subtract two matrices, they must have the same dimensions. Remember the size function tells you the dimensions of your matrix. To perform matrix addtion or subtraction, we add the individual elements of each matrix together. For $\\textbf{R} = \\textbf{S} + \\textbf{T}$, then $r_{ij} = s_{ij}+t_{ij}$\n",
    "\n",
    "## Exercise 2.1 Adding matrixes\n",
    "\n",
    "Create $\\textbf{S}$ and $\\textbf{T}$ in python and find the result $\\textbf{R}$\n",
    "\n",
    "$$\n",
    "\\textbf{S}=\n",
    "\\begin{bmatrix} \n",
    "1 & 5 & 6 \\\\\n",
    "3 & 6 & 0 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\textbf{T}=\n",
    "\\begin{bmatrix} \n",
    "8 & 2 & 3 \\\\\n",
    "4 & 1 & 6 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Create $\\textbf{S}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 5, 6],\n",
       "       [3, 6, 0]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "S = np.array([[1, 5, 6], [3, 6, 0]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create $\\textbf{T}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[8, 2, 3],\n",
       "       [4, 1, 6]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T = np.array([[8, 2, 3], [4, 1, 6]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find $\\textbf{R} = \\textbf{S} + \\textbf{T}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[9, 7, 9],\n",
       "       [7, 7, 6]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R = S + T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.2 Subtracting matrices\n",
    "\n",
    "Now try ```R-T```\n",
    "\n",
    "What is the result?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 5, 6],\n",
       "       [3, 6, 0]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R-T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Matrix multiplication\n",
    "\n",
    "To multiply a matrix by a scalar, each element in the matrix is simply multiplied by the scalar. \n",
    "\n",
    "For $\\textbf{Q} = a\\textbf{S}$, then $q_{ij} = as_{ij}$\n",
    "\n",
    "## Exercise 3.1 Pre-multiplication by a scalar\n",
    "\n",
    "If $a=10$, what answer matrix do you think the following will give\n",
    "\n",
    "```Q = a * S```\n",
    "\n",
    "Check that you were right, by first telling python that ```a = 10```:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a=10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now try ```Q = a * S```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[10, 50, 60],\n",
       "       [30, 60,  0]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Q = a * S"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Matrix multiplication involving a scalar is commutative. That is $a\\boldsymbol{S} = \\boldsymbol{S}a$, so\n",
    "\n",
    "```Q = S * a```\n",
    "\n",
    "should give the same answer as above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[10, 50, 60],\n",
       "       [30, 60,  0]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Q = S * a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.2 Pre- and post-multiplying by a scalar\n",
    "\n",
    "What do you think the following code will give:\n",
    "\n",
    "```Q = a * S * a```\n",
    "\n",
    "Check that you were right:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[100, 500, 600],\n",
       "       [300, 600,   0]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Q = a * S * a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3.3 Multiplying two vectors together\n",
    "\n",
    "To multiply a row vector by a column vector, the row vector must have the same number of columns as the column vector has rows.\n",
    "\n",
    "In Python create the following vectors:\n",
    "\n",
    "$$\n",
    "\\textbf{a}=\n",
    "\\begin{bmatrix} \n",
    "1 & 7 & 5 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\textbf{b}=\n",
    "\\begin{bmatrix} \n",
    "2 \\\\\n",
    "4 \\\\\n",
    "1\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\textbf{c}=\n",
    "\\begin{bmatrix} \n",
    "2 \\\\\n",
    "4 \\\\\n",
    "1 \\\\\n",
    "6 \\\\\n",
    "2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Make $\\textbf{a}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 7, 5]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([[1, 7, 5]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make $\\textbf{b}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2],\n",
       "       [4],\n",
       "       [1]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b = np.array([[2], [4], [1]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make $\\textbf{c}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2],\n",
       "       [4],\n",
       "       [1],\n",
       "       [6],\n",
       "       [2]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = np.array([[2], [4], [1], [6], [2]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\textbf{a}$ is a [$1$x$3$] row vector, $\\textbf{b}$ is a [$3$x$1$] column vector and $\\textbf{c}$ is a [$5$x$1$] column vector. For matrix multiplication, the 'inner' dimensions of the matrices you are multiplying must match.\n",
    "\n",
    "To find out the result of $\\textbf{ab}$ we use the code ```np.dot(a,b)```. \n",
    "\n",
    "```np.dot``` is the function for matrix multiplication."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[35]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.dot(a, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case we have [$1$x$3$] x [$3$x$1$]. The inner dimension $3$ matches.\n",
    "\n",
    "What is the result of $\\textbf{ac}$?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "shapes (1,3) and (5,1) not aligned: 3 (dim 1) != 5 (dim 0)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-24-e0b655183f59>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<__array_function__ internals>\u001b[0m in \u001b[0;36mdot\u001b[0;34m(*args, **kwargs)\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: shapes (1,3) and (5,1) not aligned: 3 (dim 1) != 5 (dim 0)"
     ]
    }
   ],
   "source": [
    "np.dot(a,c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why cant you calculate $\\textbf{ac}$?\n",
    "\n",
    "In this case [$1$x$3$] x [$5$x$1$]. The inner dimension does not match. $3$ is different to $5$\n",
    "\n",
    "The product of a row vector multiplied by a column vector is a scalar (a single number). The scalar is the sum of the first row vector element multiplied by the first column vector element plus the second row vector element multiplied by the second column vector element, and so on. If ${r} = \\textbf{ab}$, then\n",
    "\n",
    "$$\n",
    "r = \\sum_{i=1}^n a_i b_i\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\textbf{r}=\n",
    "\\begin{bmatrix} \n",
    "1 & 7 & 5 \n",
    "\\end{bmatrix}.\n",
    "\\begin{bmatrix} \n",
    "2 \\\\\n",
    "4 \\\\\n",
    "1 \\\\\n",
    "\\end{bmatrix}\n",
    "=(1*2)+(7*4)+(5*1)=35\n",
    "$$\n",
    "\n",
    "## Exercise 3.4 Multiplying two matrices together\n",
    "\n",
    "Remember, \n",
    "\n",
    "$$\n",
    "\\textbf{X}=\n",
    "\\begin{bmatrix} \n",
    "5 & 8 & 2 \\\\\n",
    "1 & 0 & 7 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and \n",
    "\n",
    "$$\n",
    "\\textbf{Y}=\n",
    "\\begin{bmatrix} \n",
    "3 & 1 \\\\\n",
    "6 & 7 \\\\\n",
    "2 & 3 \n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Calculate $\\textbf{P=XY}$ remember to use the ```np.dot``` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[67, 67],\n",
       "       [17, 22]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "P = np.dot(X, Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that: \n",
    "\n",
    "$$p_{00} = (5*3) + (8*6) + (2*2) = 67$$\n",
    "$$p_{01} = (5*1) + (8*7) + (2*3) = 67$$\n",
    "$$p_{10} = (1*3) + (0*6) + (7*2) = 17$$\n",
    "$$p_{11} = (1*1) + (0*7) + (7*3) = 22$$\n",
    "\n",
    "## Exercise 3.5 Checking if commutative\n",
    "\n",
    "Is this calculation commutative? What is $\\textbf{P=YX}$? Does this new $\\textbf{P}$ have the dimensions you expected?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[16, 24, 13],\n",
       "       [37, 48, 61],\n",
       "       [13, 16, 25]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "P = np.dot(Y, X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should have found that matrix multiplication is not commutative and you get a different answer.\n",
    "\n",
    "## Exercise 3.6 Multiplying by the identity matrix\n",
    "\n",
    "Calculate $\\textbf{P=PI}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[16., 24., 13.],\n",
       "       [37., 48., 61.],\n",
       "       [13., 16., 25.]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "P = np.dot(I, P)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the identity matrix acts like 1. Pre-multiplying or post-multiplying by the identity matrix has no effect.\n",
    "\n",
    "# 4. Matrix transpose\n",
    "\n",
    "Transposing a matrix swaps the rows to be columns, or turns a row vector into a column vector (or vice-versa). In algebra this is denoted by the apostrophe:  $\\textbf{P'}$ is the transpose of $\\textbf P$.\n",
    "\n",
    "## Exercise 4.1 Using transpose in python\n",
    "\n",
    "In python try ```np.transpose(P)```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[16., 37., 13.],\n",
       "       [24., 48., 16.],\n",
       "       [13., 61., 25.]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.transpose(P)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Matrix inverse\n",
    "\n",
    "The inverse of a number is the number which, when multiplied by the original number, gives a product of 1. Hence the inverse of $4$ is $\\frac{1}{4}$, because $4 * \\frac{1}{4} = 1$. The inverse of $x$ is $\\frac{1}{x}$ or $x^{-1}$.\n",
    "\n",
    "In matrix algebra, the inverse of a matrix is that matrix which, when multiplied by the original matrix, gives the identity matrix, so: $${\\textbf{A}\\textbf{A}^{-1} = \\textbf{I}}$$\n",
    "\n",
    "## Exercise 5.1 Inverting a matrix in Python\n",
    "\n",
    "Make \n",
    "\n",
    "$$\n",
    "\\textbf{X}=\n",
    "\\begin{bmatrix} \n",
    "2 & 0 & 1\\\\\n",
    "3 & 1 & 1\\\\\n",
    "4 & 2 & 3 \n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 0, 1],\n",
       "       [3, 1, 1],\n",
       "       [4, 2, 3]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.array([[2,0,1],[3,1,1],[4,2,3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And use the ```np.linalg.inv()``` function to find $\\textbf{X}^{-1}$\n",
    "\n",
    "You can call this variable ```invX``` if you like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.25,  0.5 , -0.25],\n",
       "       [-1.25,  0.5 ,  0.25],\n",
       "       [ 0.5 , -1.  ,  0.5 ]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "invX = np.linalg.inv(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 5.2 Multiplying a matrix by its inverse\n",
    "\n",
    "What is the result of $\\textbf{X}\\textbf{X}^{-1}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 0., 0.],\n",
       "       [0., 1., 0.],\n",
       "       [0., 0., 1.]])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.dot(X, invX)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Element-wise multiplication and division\n",
    "\n",
    "Another type of matrix product is the element-wise product. In this case, each $i$, $j$ element of $\\textbf{A}$ is multiplied with each $i$, $j$ element of $\\textbf{B}$. In other words, the element in the first row, first column of $\\textbf{A}$ is multiplied with the element in the same position of $\\textbf{B}$ to form the element in the new result matrix. \n",
    "\n",
    "This is <b>different</b> to the matrix multiplication discussed in section 3. In matrix algbra, this element-wise multiplication is written as $\\textbf{A} \\circ \\textbf{B}$ and is called the Hadamard product.\n",
    "\n",
    "## Exercise 6.1 Element-wise multiplication of matrices\n",
    "\n",
    "In Python create\n",
    "\n",
    "$$\n",
    "\\textbf{Y}=\n",
    "\\begin{bmatrix} \n",
    "5 & 1 & 0\\\\\n",
    "1 & 3 & 7\\\\\n",
    "4 & 2 & 3 \n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5, 1, 0],\n",
       "       [1, 3, 7],\n",
       "       [4, 2, 3]])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y = np.array([[5,1,0],[1,3,7],[4,2,3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use ```np.multiply(X,Y)``` to find $\\textbf{X} \\circ \\textbf{Y}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[10,  0,  0],\n",
       "       [ 3,  3,  7],\n",
       "       [16,  4,  9]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.multiply(X,Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now try ```X * Y```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[10,  0,  0],\n",
       "       [ 3,  3,  7],\n",
       "       [16,  4,  9]])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X * Y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Does this simple multiplication give true matrix multiplication or the Hadamard product?\n",
    "\n",
    "Be very careful that you chose the correct type of matrix multiplication function in our work with matrices. Most of the time we will be working with ```np.dot()```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 6.2 Element-wise division of matrices\n",
    "\n",
    "Try ```np.divide(X,Y)``` and try ```X/Y```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "//anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:1: RuntimeWarning: divide by zero encountered in true_divide\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[0.4       , 0.        ,        inf],\n",
       "       [3.        , 0.33333333, 0.14285714],\n",
       "       [1.        , 1.        , 1.        ]])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X/Y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we are having a problem dividing 1 by zero, leaving an infinity in the top right cell.\n",
    "\n",
    "We'll talk about how to deal with this in a later tutorial.\n",
    "\n",
    "# 7. Further matrix operations in Python\n",
    "\n",
    "## Exercise 7.1 Sums\n",
    "\n",
    "What do the following commands do ```np.sum(X,0)```, ```np.sum(X,1)``` and ```np.sum(np.sum(X))```?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(np.sum(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In python matrices we refer to cells by giving the the row location followed by the column location. Think of this as \"how many down, then how many across\". In the ```np.sum``` function, the specifier ```0``` has the effect of summing 'down' and the specifier ```1``` has the effect of summing 'across'. ```np.sum(np.sum))``` has the effect of summing in both dimensions to give the total of the whole matrix\n",
    "\n",
    "## Exercise 7.2 Extracting a single row from a matrix\n",
    "\n",
    "To select the first row of $\\textbf{X}$ we use the command ```X[0,:]``` This means take the first row and every column element. What do you get if you use the command ```X[:,1]```?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[:,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 7.3 Extracting a single column from a matrix\n",
    "\n",
    "Can you write a single command to find the sum of the 3rd column of $\\textbf{X}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(X[:,2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 7.4 Repeated rows\n",
    "\n",
    "Find ```t = np.sum(X,0)```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([9, 3, 5])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = np.sum(X, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we wanted to find the proportion that each element in $\\textbf{X}$ was of the total column sum, we’d want to divide each element in $\\textbf{X}$ by the corresponding column sum in $\\textbf{t}$. But to use element-wise division, the dimensions of each matrix must be the same. In this case $(3,3)$. What we need to do is make a new matrix $\\textbf{T}$, which has 3 rows each containing the original $\\textbf{t}$ and use this in our division calculation. The command:\n",
    "\n",
    "```python\n",
    "T = np.tile(t,[3,1])\n",
    "```\n",
    "\n",
    "makes a new matrix $\\textbf{T}$ by repeating original $\\textbf{t}$ down three rows but does not repeat across any more columns. <b>We will be using this concept in workbook 2</b>. Try it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[9, 3, 5],\n",
       "       [9, 3, 5],\n",
       "       [9, 3, 5]])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T = np.tile(t,[3,1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can divide $\\textbf{X}$ by  $\\textbf{T}$ to give a new matrix showing what proportion each element is of the column total. <b>We will be using this concept in workbook 2</b>. Yes I've said this twice. Try \n",
    "\n",
    "```prop = X/T```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.22222222, 0.        , 0.2       ],\n",
       "       [0.33333333, 0.33333333, 0.2       ],\n",
       "       [0.44444444, 0.66666667, 0.6       ]])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prop = X/T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Key learning points\n",
    "\n",
    "You should have learnt:\n",
    "\n",
    "<ol>\n",
    "<li>What matrices, vectors and scalars are</li>\n",
    "<li>How to make matrices in Python using the np.array function</li>\n",
    "<li>What the conditions are for adding and subtracting matrices</li>\n",
    "<li>What the two types of matrix multiplication are</li>\n",
    "<li>What the conditions are for matrix multiplication</li>\n",
    "<li>What the inverse of a matrix means</li>\n",
    "<li>What the identity matrix is and how it works</li>\n",
    "<li>How to make diagonal matrices and matrices of zeros</li>\n",
    "<li>How to refer to matrix rows, columns and individual elements</li>\n",
    "<li>How to sum matrices along the rows and along the columns</li>\n",
    "<li>How the tile function works</li>\n",
    "   \n",
    "</ol>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
